Background: Information and communication technology (ICT) has been used with increasing frequency for the assessment of diet and physical activity in health surveys. A number of Web-based questionnaires have been developed for children and adolescents. However, their usability characteristics have scarcely been reported, despite their potential importance for improving the feasibility and validity of ICT-based methods.
Objective: The objective of this study was to describe the usability evaluation of the Consumo Alimentar e Atividade Física de Escolares (CAAFE) questionnaire (Food Consumption and Physical Activity Questionnaire for schoolchildren), a new Web-based survey tool for the self-assessment of diet and physical activity by schoolchildren.
Methods: A total of 114 schoolchildren aged 6 to 12 years took part in questionnaire usability testing carried out in computer classrooms at five elementary schools in the city of Florianopolis, Brazil. Schoolchildren used a personal computer (PC) equipped with software for recording what is on the computer screen and the children’s speech during usability testing. Quantitative and qualitative analyses took into account objective usability metrics such as error counts and time to complete a task. Data on the main difficulties in accomplishing the task and the level of satisfaction expressed by the children were assessed by the observers using a standardized form and interviews with the children. Descriptive statistics and content analysis were used to summarize both the quantitative and the qualitative aspects of the data obtained.
Results: The mean time for completing the questionnaire was 13.7 minutes (SD 3.68). Compared to the children in 2nd or 3rd grades, those in 4th or 5th grades spent less time completing the questionnaire (median 12.4 vs 13.3 minutes, P=.022), asked for help less frequently (median 0 vs 1.0 count, P=.005), had a lower error count (median 2.0 vs 8.0 count, P<.001), and obtained a higher overall performance score (median 73.0 vs 68.0, P=.005). Children with a PC at home spent less time completing the questionnaire (median 12.3 vs 14.9 minutes, P<.001), had a lower overall error count (median 2.0 vs 9.0 count, P=.03), and had a higher performance score (median 72.0 vs 64.0, P=.005) compared to the children without a PC at home. The most common difficulty in completing the questionnaire was in using the scroll bar. The majority of children reported a positive evaluation (liked a lot or liked) for the four design elements, which were evaluated.
Conclusions: The results of the present study provided feedback to improve the final version of the CAAFE questionnaire. Quantitative data showed minor errors and system failures, while qualitative data indicated that, overall, the children enjoyed the CAAFE questionnaire. Grade levels and PC use must be taken into account in Web-based tools designed for children.